A no-reference objective image quality metric based on perceptually weighted local noise

This work proposes a perceptual based no-reference objective image quality metric by integrating perceptually weighted local noise into a probability summation model. Unlike existing objective metrics, the proposed no-reference metric is able to predict the relative amount of noise perceived in images with different content, without a reference. Results are reported on both the LIVE and TID2008 databases. The proposed no-reference metric achieves consistently a good performance across noise types and across databases as compared to many of the best very recent no-reference quality metrics. The proposed metric is able to predict with high accuracy the relative amount of perceived noise in images of different content. Introduction Reliable assessment of image quality plays an important role in meeting the promised quality of service (QoS) and in improving the end user’s quality of experience (QoE). There is a growing interest to develop objective quality assessment algorithms that can predict perceived image quality automatically. These methods are highly useful in various image processing applications, such as image compression, transmission, restoration, enhancement, and display. For example, the quality metric can be used to evaluate and control the performance of individual system components in image/video processing and transmission systems. One direct way to evaluate video quality is through subjective tests. In these tests, a group of human subjects are asked to judge the quality under a predefined viewing condition. The scores given by observers are averaged to produce the mean opinion score (MOS). However, subjective tests are time-consuming, laborious, and expensive. Objective image quality (IQA) assessment methods can be categorized as full reference (FR), reduced reference (RR), and no reference (NR) depending on whether a reference, partial information about a reference, or no reference is used for calculation. Quality assessment without a reference is challenging. A no-reference metric is not relative to a reference image, but rather an absolute value *Correspondence: [email protected] School of Electrical, Computer & Energy Engineering, Arizona State University, Tempe, AZ 85287, USA is computed based on some characteristics of the test image. Of particular interest to this work is the no-reference noisiness objective metric. Noisiness and blurriness are two key distortions in multiple applications, and typically there is a tradeoff to balance between noisiness and blurriness. For example, in soft-thresholding for image denoising [1], the image could be blurry when the threshold is high, while the image could remain noisy when the threshold is low. Also, in Wiener-based super-resolution [2], too much regularization will result in less noise at the expense of more blur. The reconstructed image could be blurry when the auto-correlation function is modeled to be too flat, while the reconstructed image could be noisy when the auto-correlation function is modeled to be too sharp. No-reference image sharpness/blur metrics have been widely discussed [3,4]. However, these image sharpness/blur metrics typically fail in the presence of noise. The sharpness metric may increase when noise increases. A no-reference noise-immune image sharpness metric was also proposed [5]. Furthermore, all the edge-based sharpness metrics can be easily applied in the wavelet domain as described in [5] to provide resilience to noise. Still, it lacks the ability to assess the impairment due to noise. For visual quality assessment of noisiness, many full-reference metrics are presented in [6], such as peak signal-to-noise ratio (PSNR), multi-scale structural similarity (MS-SSIM) [7], noise quality measure (NQM) [8], and information fidelity criterion (IFC) [9]. © 2014 Zhu and Karam; licensee Springer. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Zhu and Karam EURASIP Journal on Image and Video Processing 2014, 2014:5 Page 2 of 8 http://jivp.eurasipjournals.com/content/2014/1/5 However, these full-reference metrics require the reference image for calculation. There is a need to develop a no-reference noisiness quality metric. Furthermore, such noisiness metric could be further combined with the noreference blur metrics [3,4] to provide a better prediction of image quality for several applications including superresolution, image restoration, and othermultiply distorted images. A global estimate of image noise variance was used as a no-reference noisiness metric in [10]. The histogram of the local noise variances is used to derive the global estimate. However, the locally perceived visibility of noise is not considered. Similarly in [11], noisiness is expressed by the sum of estimated noise amplitudes and the ratio of noise pixels. Both the metrics of [10,11] do not account for the effects of locally varying noise on the perceived noise impairment and they do not exploit the characteristics of the human visual system (HVS). To tackle this issue, this paper firstly presents a fullreference image noisiness metric which integrates perceptually weighted local noise into a probability summation model. This proposed metric can predict the perceptual noisiness in images with high accuracy. In addition, a noreference objective noisiness metric is derived based on local noise standard deviation, local perceptual weighting, and probability summation. The experimental results show that the proposed FR and NR metrics show better and more consistent performance across databases and distortion types, when compared with several very recent FR and NR metrics. The remainder of this paper is organized as follows. A perceived noisiness model based on probability summation is presented first followed by details on the contrast sensitivity thresholds computation. A full-reference perceptually weighted noise (FR-PWN) metric is proposed next based on perceptual weighting using the computed contrast sensitivity thresholds and probability summation. After that, a no-reference perceptually weighted noise (NR-PWN) metric is further derived. Performance results and comparison with existing metrics are presented followed by a conclusion. Perceptual noisiness model based on probability summation The PSNR simply calculates the difference point by point. However, the human visual system should be taken into consideration since the visual impairment due to the same noise could be perceived differently based on the local characteristics of the visual content. Contrast is a key concept in vision science because the information in the visual system is represented in terms of contrast and not in terms of the absolute level of light. So, the relative changes in luminance are important rather than the absolute ones [3]. The contrast sensitivity threshold measures the smallest contrast or the just-noticeable difference (JND) that yields a visible signal over a uniform background. The proposed metric makes use of JND for calculating the probability of noise detection. Even when the noise is uniform, the impact of the noise will be more visible in image regions with a relatively lower JND. Consider the noisy signal y as y(i, j) = y′(i, j)+ error(i, j) (1) where y′(i, j) is the original undistorted image. The probability of detecting a noise distortion at location (i, j) can be modeled as an exponential having the following form P(i, j) = 1− exp ( − ∣∣∣error(i, j) JND(i, j) ∣∣∣ β )